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1 Introduction 

In what follows, we denote scalars by lowercase letters {a,b,c,...), vectors by 
boldface lowercase letters (a, b, c, ...) and matrices by boldface uppercase letters 
(A, B, C,. ..). Greek letters (a, / 3 ,7,...) are reserved for particular scalar quan¬ 
tities. We denote by A^ the matrix transpose of A and by || • || the Euclidean 
norm. 


1.1 ICA Data Model 

We consider the following noiseless linear ICA model: 

y(t) = Hs(t), 


where 

1. s{t) (si(f),..., Sd(t))^ denotes the tth realization of the unknown 
source signal. The components Si(f),... ,Sd(t) are mutually statistically 
independent, have unit variance and at most one of them is Gaussian. 
Furthermore, s(l),. .. ,s{N) denote N independent realizations of s. 

2. y(t) (?/i(t), ■ ■ • ,yd{t)y denotes the tth realization of the observed sig¬ 
nal 

3 . H e is a full rank square matrix, called the mixing matrix. 


1.2 Data Preprocessing 


Most ICA methods require the observed signal {y(t)} to be standardized |ll2l3j . 
The standardization of {y(t)} consists of the data centering and data whitening, 
which involve the estimation of E[y] and Cov(y). In practice, E[y] and Cov(y) 
are usually estimated by the sample mean and sample variance: 

N N 

t=i ^ ' t=i 

In this work, we shall consider several different data preprocessing scenarios. 
Denote 


~ 1 

t=i 

The following data preprocessing scenarios will be studied: 

1) . Theoretical whitening and theoretical centering. 

x(t)‘'A'Cov(y)-5(y(t)-E[y]). (1) 

2) . Theoretical whitening and empirical centering. 

x(t)1i'Cov(y)-^(y(t)-y). (2) 

3) . Empirical whitening and theoretical centering. 

X(t) =''C"5(y(t) _E[y]). (3) 

4) . Empirical whitening and empirical centering. 

x(t)1i'c-^(y(t)-y). (4) 

In the seqnel, x(t) will always stand for the standardized signal under one of the 
scenarios defined above. The specific data preprocessing scenario will be stated 
explicitly when necessary. 


1.3 Variants of the FastICA Algorithm 

Before proceeding further, we need to introduce some notations first. We denote 
by S the unit sphere in K.'^. We denote by g(-) : R —>■ M the nonlinearity function, 
and by G(-) its primitive. The nonlinearity function g is usually supposed to be 
non-linear, non-quadratic and smooth. Eor any function / : R'^ —> R™, we write 
Ex[/(x)] A /(x(t)) for conciseness. 




The Deflationary FastICA Algorithm This version of the FastICA algo¬ 
rithm extracts the sources sequentially. It consists of the following steps [3] : 

- Input: x(l),..., x(A). 

1) . Set p = 1. 

2) . Choose an arbitrary initial iterate w G S; 

3) . Run iteration 


W G- Ex[p^(w'''x)w — p(w'''x)x] 

(5) 

p-1 


w ^ -w — 

(6) 



w 

(7) 

w ■<—- 

IN,rll 


until convergence^. The limit is stored as 

4) Break ii p = d. Otherwise p t— p -I- 1 then go to step 2). 

- Output: ..., 

The Symmetric FastICA Algorithm The symmetric version of FastICA 
extracts all the sources simultaneously. It can be described as follows: 

- Input: x(l),..., x(A). 

1) . Choose an arbitrary orthonormal matrix W = (wi, ..., w^) G 

2) . Run 


Wi ^ Ex[p'(w7x)wi - sr(w7x)x] 

(8) 

Wd ^ Ex[p'(wjx)vifi - p(w7x)x] 

(9) 

/ -r\-l/2 


W ^ f WW^ j W 

(10) 


until convergence. The limit is denoted by 

- Output: ..., 


2 Asymptotic Performance 

Let us introduce the notion of gain matrix: 

qDFL dg (y^DFLyfj~l/2-^j_^ qSYM dg 

where stands for the sphering matrix used in the data preprocessing stage, 

i.e. C = Cov(y) in scenarios ([T]) and ([2]), C = C in scenario ([3]) and C — C 
in scenario ®. Without loss of generality, we shall omit the permutation and 


^ We impose the number of iterations to be even, so that the well known sign-flipping 
phenomenon disappears. 




sign ambiguities of ICA. Then, « I and ss I, hence 

and can be considered as estimators of B (H~^)^. In the 

sequel, we will study the asymptotic errors of — B) and 

^i/2(c-i/2W'^^^ — B) under proposed data preprocessing scenarios. 

The proofs of the results presented below are based on the method of M- 
estimators. However, all proofs will be omitted due to the lack of space. A com¬ 
plete version of this work can be provided upon request. The readers are also 
referred to [1] for a more detailed account of this subject. 

2.1 The Asymptotic Error of Deflationary FastICA 

Assume that the following mathematical expectations exist for i = 1,..., d: 


Ui = "^g'{zi) - g{zi)zi] 


7 * ¥\g{zi)zi] 



where Zi = Si — E[si] for z = 1 ,..., d. 

Theorem 1. Let denote the ith column o/B. Under some mild regularity 
conditions, we have 



where k G {1,2, 3,4} is the label of the underlying data preprocessing scenario 
(see and is given as follows: 
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Corollary 2. There holds -5ij) A/'(0, where 

denotes the {i,j)th entry of and is given as follows: 

1. Case j < i: 


2. Case j = i: 
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3. Case j > i: 
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2.2 The Asymptotic Error of Symmetric FastICA 

Theorem 3. Under some mild regularity conditions, we have — 

hi) —-—>■ A/’(0, RfX^), where 
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Corollary 4. Fori,j = 1,..., d, there holds —Sij) —>■ A/’(0, 

where 

1. Case j = i: 


^rSYM _ jrSYM _ n i/SYM _ irSYM _ 

\/(l) - V/( 2 ) - U, \/(3) - »/(4) - 

2. Case j ^ z; 

ySYM _ Pi + Pi — “^lilj 

" (l«.| + l«.l)" ’ 
ySFM ^ Pi + /3j- - ‘^l^li - ‘^if 
(Kl + I«.l)^ ’ 

ySYM ^ P^ - if + P 3 - if + 

(3) (|ad + l«,l)^ ’ 

SYM ^ P^ - if + P 3 - Ij + 
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(25) 
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Remark 5. Although the asymptotic error of the FastICA algorithm has already 
been studied by quite a few researchers mm. many of the results presented 
in this contribution, notably expressions (rTT|) - (fT51) established in Theorem[l]and 
((Illl-dia) in Theorem [21 are new. 

Example 1. The validity of formulas (fT5l) - (IT2)) and (l^ - (E51) is verified in com¬ 
puter simulations, see Fig.[T]and Fig.j^l The simulations are configured as follows: 
d = 3, N = 5000, all three sources have identical bimodal Gaussian distribution 
with asymmetrical density. Both deflationary FastICA and symmetric FastICA 
have been tested with different data preprocessing 0-0 in 5000 independent 
trials. 


2.3 Discussion 


First, comparing the expressions in Corollary [2] and CorollarySl we find that for 
the (z, j)th entry of the gain matrix, 


-\/DFL TrDFL _ ^rDFL -\/DFL 
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Since all the differences above are non-negativcQ, we assert that the empirical 
data centering generally leads to a better asymptotic performance. 

3 Conclusion 

The contribution of this work is twofold. First, we derived explicit formulas 
for the asymptotic error of the two most important variants of the FastICA 
algorithm, the deflationary FastICA and the symmetric FastICA, under four 
different data preprocessing scenarios. Many of the presented formulas are novel. 
Second, we assessed the impact of empirical data preprocessing procedure on 
the asymptotic performance of the algorithms. We showed that, compared to 
the theoretical data centering, the empirical data centering generally leads to a 
better separation performance. 
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They become zero if gi and/or gj vanish. This is the case if, e.g. g is pair and the 
involved sources have symmetric distributions. 
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Fig. 1. Asymptotic error of the deflationary FastICA in each preprocessing scenario. 
We plotted the histograms of an (upper) off-diagonal entry of in 5000 

independent trials versus the theoretical curves of the Gaussian PDFs with variances 
given by (HSJ-IHJ. 



Fig. 2. Asymptotic error of the symmetric FastICA in each preprocessing scenario. We 
plotted the histograms of an off-diagonal entry of 5 qqq independent trials 

versus the theoretical curves of the Gaussian PDFs with variances given by (1231)-®- 










































































